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tl^p spatial relationship in the panoramic region of the 
^ content of the displayed image to the content of the other 

displayed images ; and 

nstructions for displaying one or more context images 
while ^n active image is displayed, each context image being 
at leasts part of an image represented by a frame of the set 
of temporally sequential frames of visual data. 



Please enter the igilowing new claim: 



ft 



25. (New) A method as in Claim 13, wherein: 

the set of frames of visual data in which^-^bfanch 
point is identified is a first sej>HsriTtemporally sequential 
frames of visual data rp^fesenting a panoramic region at a 
first time; and^ 

th^^rfew set of frames of visual data is a second set of 
te^orally sequential frames of visual data representing the 
panoramic region at a second time that is different from the 
first time. 



REMARKS 

Claims 1-24 were filed and are pending. Claims 1-24 were 
rejected under 35 U.S.C. § 102. Claims 1, 23 and 24 have been 
amended. Claim 25 has been added. Reconsideration and allowance 
of Claims 1-24, and allowance of Claim 25 is requested. 
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Rejection of Claims under' 35 U'.S.C. § 102 



The Examiner rejected Claims 1-24 under 35 U.S.C. § 102(b) 
as being anticipated by Kumar et al . (U.S. Patent No. 5,963,664). 
Since the Kumar et al'. patent issued on October 5, 1999, and the 
instant application was filed on March 31, 1999, the Kumar et al . 
patent cannot be used to reject Claims 1-24 under 
35 U.S.C. § 102(b). Applicants have assumed herein that the 
Examiner meant to reject Claims 1-24 under 35 U.S.C. § 102(e) as 
oeing anticipated by Kumar et al . 

Regarding Claims 1, 18, 23 and 24, the Examiner stated: 

Kumar discloses (Fig. 2) a set of temporally sequential 
frames of visual data, each frame representing an 
image, defining a region, within a panoramic region. 
Further disclosed (Fig. 2) is the positioning of one or 
more images, with relationship to the spatial 
orientation of the panoramic. region, while an active 
image is being displayed. In addition, (Fig. 1, 
Element 106) shows an output device, which could be any 
type of video di'splay. 



Further disclosed in Claim 24, is a readable storage 
medium, on which is stored one or more computer 
programs, for producing a visual display, of a 
panoramic region. Kumar discloses (Col. 4, Lines 58- 
61) and (Col. 5, Lines 15-17) a computer that is 
programmed to function as an image processing system, 
and a video storage and retrieval system. 

As stated in Applicants' specification at page 7, lines 8- 
11, n [t]he invention provides a visual display of a panoramic 
region in which visual context (a ^context display, ' . . . ) is 
provided for an active display . . . displayed in a mobile display 
window." As further stated in Applicants' specification at 
page 1 1 , lines 1 7-27 :*« 
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After the display of a first image, successive images 
(active images) are generated and displayed from frames 
of visual data that are temporally adjacent to the 
frame of visual data from which the most recently 
displayed image was generated. If the frames of visual 
data have been obtained and are displayed at a 
sufficiently high rate, successive display of active 
images will produce a display in which it appears that 
motion occurs (if motion was occurring in the part of 
the panoramic region represented by the active image) 
in the active window (i.e., the area of the display 
screen in which the active image is displayed). 

As further stated in Applicants' specification at page 12, 

line 34 to page 13, Line 9: 

[A]s active images are successively displayed, one or 
more context images (i.e., a complete or partial 
previously displayed image) can be retained as part of 
the visual display. Each context image is static. In 
contrast, as described above, motion can appear to 
occur in the active window in which the active image is 
displayed. Thus, in a visual display according to the 
invention, one or more static images provide context 
for an active image that, in many cases (depending upon 
whether motion occurred at that time in that part of 
the panoramic region), is a moving image. 

Kumar et al . do not teach or suggest a visual display of a 
panoramic region in which visual context (a context display) is 
provided for an active display, as in Claims 1, 23 and 24 
originally pending in this application. Nevertheless, Claims 1, 
23 and 24 have been amended herein to emphasize and/or clarify 
aspects of the invention not taught by Kumar et al. Claims 1, 23 
and 24 each recite, as amended, "displaying a series of active 
images generated from multiple temporally sequential frames of 
visual data" and "displaying one or more context images while an 
active image is displayed, each context image being at least part 
of an image represented by a frame of the set of temporally 
sequential frames of visual data." Claim 18 recites "an active 
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display comprising a series of active images that are generated 

from temporally sequential frames of visual data" and "a context 

display comprising one or more context images displayed while an 

active image is displayed, each context image being at least part 

of an image represented by a frame of the set of temporally 

sequential frames of visual data." 

Kumar et al. teach, at column 3, lines 61-66: 

The present invention is an image processing system 
that combines a plurality of images representing an 
imaged scene to form a three-dimensional (3D) mosaic, 
where the 3D mosaic contains an image mosaic 
representing a panoramic view of the scene and a shape 
mosaic representing the three-dimensional geometry of 
the scene. 

Kumar et al. further teach, at column 5, lines 18-32: 

FIG. 2 schematically depicts the input images 200 n to 
the system of Fib. 1 and the output 3D mosaic 202 
generated by that system in response to the input 
images. The input images, as mentioned above, are a 
series of images of a scene, where each image depicts 
the scene from a different viewpoint. The system 
aligns the images and combines them to form an image 
mosaic 204, e.g., a two-dimensional mosaic having the 
images aligned along an arbitrary parametric surface 

extending through all the images In addition to 

the image mosaic, the system generates a shape mosaic 
206 that contains the motion field that relates the 
three-dimensional objects within the images to one 
another and to the parametric surface 

Thus, Kumar et al. teach a particular way of creating a 

static panoramic display including three-dimensional information. 

Kumar et al. do not teach or suggest "an active display 

comprising a series of active images that are generated from 

temporally sequential 1 frames of visual data," as recited in 

Claim 18, or "displaying a series of active images generated from 

multiple temporally sequential frames of visual data," as recited 
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in Claims 1, 23 and 24. Nor do Kumar et al . teach or suggest 
providing context for such an active display, i.e., Kumar et al . 
do not teach or suggest "a context display comprising one or more 
context images displayed while an active image is displayed, each 
context image being at least part of an image represented by a 
frame of the set of temporally sequential frames of visual data," 
as recited in Claim 1*8, or "displaying one or more context images 
while an active image is displayed, each context image being at 
least part of an image represented by a frame of the set of 
temporally sequential frames of visual data," as recited in 
Claims 1, 23 and 24. Therefore, Claims 1, 18, 23 and 24 are 
allowable over the teaching of Kumar et al . 

Claims 2-17 each depend, either directly or indirectly, on 
Claim 1 and are therefore allowable over the teaching of Kumar et 
al . for at least the reasons given above with respect to Claim 1. 
Similarly, Claims 19-22 each depend on Claim 18 and are therefore 
allowable over the teaching of Kumar et al . for at least the 
reasons given above with respect to Claim 18. Additionally, as 
discussed below, many of the limitations recited in Claims 2-17 
and 19-22 are neither taught nor suggested by Kumar et al . 

Regarding Claim 2, the Examiner stated: 

Claim 2 discloses one or more context images, being 
automatically displayed, in a predetermined manner. 
Kumar teaches (Fig. 7), (Col. 11, Lines 1-3) a 
hierarchical technique to register images. 

Claim 2 recites that "the step of displaying one or more 

context images further comprises automatically displaying one or 
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more context images in a predetermined manner." As stated in 

Applicants' specification at page 4, lines 26-31: 

In one embodiment of the invention, context images are 
automatically displayed in a predetermined manner. For 
example, after each image is displayed as the active 
image, the image can be retained, in whole or in part, 
on the display screen as a context image, i.e., the 
active image leaves a trail of context images. 

"Registering images," as used by Kumar et al . , refers to aligning 

images to create a "seamless 11 panoramic display and computing 

information regarding three-dimensional characteristics of the 

display (see, e.g., column 2, line 51 to column 3, line 4, 

column 4, lines 12-18, and column 6, lines 26-31). The 

hierarchical technique mentioned by Kumar et al . at column 11, 

line 1 is simply one example of such registration. This clearly 

does not teach or suggest the limitations recited in Claim 2. 

Regarding Claims 3 and 4, the Examiner stated: 

Claims 3-4 disclose an order, in which context images 
are to be displayed. Since the frames representing an 
image are processed sequentially, and a hierarchy for 
how those images are displayed, already exists, it is 
logical to assume that the context images would be 
displayed, as disclosed in Claims 3-4. 

The Examiner appears not to have understood the limitations 
of Claims 3 and 4. Claim 3 recites that "each context image is 
at least part of an active image displayed prior to the display 
of a current active image." Claim 4 recites that "the step of 
displaying one or more context images further comprises 
displaying each image displayed after a first display of a 
context image." Thus, contrary to the Examiner's assertion, 
Claims 3 and 4 do not recite an order in which context images are 
to be displayed. To the extent the remainder of the Examiner's 
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remarks can be understood; those remarks do not appear to bear on 
the patentability of Claims 3 and 4. In any event, since Kumar 
et al . do not teach a display in which a series of images are 
displayed, Kumar et al . clearly do not teach or suggest the 
limitations recited in Claims 3 and 4. 

i 

Regarding Claims 5-12, the Examiner stated: 

Kumar discloses (Col. 4, Lines 58-61) and (Col. 5, 
lines 15-17) a computer that is programmed to function 
as an image processing system, and a video storage and 
retrieval system. The use of computer software, and a 
memory, implies that past and present images can be 
designated, by user instruction, to perform all 
functions disclosed in Claims 5-12. 

The Examiner appears not to have understood the limitations 
of Claims 5-12 and, in any event, has provided an inadequate and 
dismissive rationale for rejecting those claims. Moreover, 
contrary to the Examiner's assertion, the use of computer 
software and a memory does not in any way imply that past and 
present images can be designated by user instruction, and, in any 
event, such an assertion does not address the particular 
limitations recited in Claims 5-12. 

Claim 5 recites that "the step of displaying one or more 
context images further comprises displaying an image designated 
by a user as a context image" and Claim 7 recites that the method 
"further compris[es] the step of designating an image as a 
context image in accordance with user instruction." Since Kumar 
et al. do not teach displaying a "context image," as that term is 
used with respect to Applicants' invention, Kumar et al. do not 
teach or suggest user designation of a context image, as recited 
in Claims 5 and 7. Nor do Kumar et al . teach or suggest the 




particular aspects of designating and displaying a context image 
recited in Claims 6, 8 and 9. For example, since Kumar et al. do 
not teach a display in which a series of images are displayed, 
Kumar et al . clearly do not teach or suggest that "the step of 
displaying one or more context images further comprises 
displaying a previously displayed image designated by a user as a 
context image," as recited in Claim 6, or that "the step of 
designating further comprises designating an image as a context 
image at the time that the image is displayed as an active 
image," as recited in Claim 8. Additionally, Kumar et al . do not 
teach or suggest "moving the location of a context image on the 
display screen in accordance with user instruction," as recited 
in Claim 9 (and the Examiner has given no indication where such a 
limitation is taught or suggested by Kumar et al.). 

Claim 10 recites that "one or more of the context images is 
designated as a clip image." Claims 11 and 12 recite further 
limitations regarding the use of a "clip image" in a method of 
Applicants' invention. As stated in Applicants' specification at 
page 21, lines 28-31, "'clip image' indicates a context image 
which, if 'selected' by a user, causes a display of successive 
active images (a 'clip'), beginning with the clip image, for a 
prescribed amount of time." The Examiner has not in any way 
indicated how Kumar et al . teach or suggest the use of a "clip 
image" (which, as can be seen, has a particular meaning with 
respect to Applicants' invention) and, in fact, Kumar et al . do 
not teach or suggest the use of a clip image. 
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Regarding Claim 13, the Examiner stated: 

Kumar (Fig. 1) also shows the use of multiple cameras, 
which would provide at least two non-coincidental 
images, as well as a plurality of sets of temporally 
sequential frames, as further disclosed in Claim 13. 
Also disclosed i-p. Claim 13 is the identification of a 
branch point, for the purpose of updating an image. 
Kumar discloses (Col. 17, Lines 13-33) a scene change 
detection scheme. 

The Examiner appears not to have understood the limitations 

of Claim 13. A scene change detection scheme is irrelevant to 

consideration of the na tentabili ty of the limitations of 

Claim 13. Claim 13 recites "identifying a branch point in a set 

of frames of visual data from which active images are being 

displayed; when a branch point is identified, determining whether 

to display images generated from a new set of frames of visual 

data; if images are to be generated from a new set of frames of 

visual data, identifying a new set of frames of visual data and 

the frame within the new set of frames of visual data with which 

to begin the display of active images; and generating a display 

of an active image from each of multiple temporally sequential 

frames of visual data in the new set of frames of visual data." 

As stated in Applicants' specification at page 24, lines 5-13: 

[A]t particular points ("branch points") within [a] set 
of frames of visual data, further display of active 
images can continue using the frames of that set of 
frames of visual data or using the frames of a 
different set of frames of visual data. If the latter 
is chosen (either automatically or at the instruction 
of a user), the active display continues using a frame 
of visual data corresponding to a predefined point 
(also referred to as a "branch point") within that set 
of frames of visual data. 

As can readily be appreciated, the Examiner has not in any way 

i 

indicated how Kumar et al. teach or suggest the use of a branch 
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point as recited in Claim' 13. In fact, since, as discussed 

above, Kumar et al . do not teach or suggest the generation of an 

active display, Kumar et al. clearly do not teach or suggest the 

use of a branch point as a mechanism for switching between 

different active displays, as in Claim 13. 

Regarding Claims 14 and 19, the Examiner stated: 

Kumar teaches (Fig. 2, Element 204) that each 
overlapping section, of an image, is updated by the new 
image . 

The fact that im ( ages can be displayed, while being 
produced, suggests that an active image is produced in 
real time. 

The Examiner's statement that "Kumar teaches (Fig. 2, 
Element 204) that each overlapping section, of an image, is 
updated by the new image" indicates a misunderstanding of the 
teaching of Kumar et al. Fig. 2 of the Kumar et al . patent (and 
the associated description) does not concern updating an image. 
Rather, Fig. 2 depicts (among other things) an image mosaic 204. 
As stated by Kumar et al. at column 5, lines 23-26, "[t]he system 
aligns the images and combines them to form an image mosaic 204, 
e.g., a two-dimensional mosaic having the images aligned along an 
arbitrary parametric surface extending through all the images." 
Further, since, as discussed above, Kumar et al. do not teach or 
suggest the display of a context image as in Applicants' 
invention, Kumar et al . clearly do not teach or suggest that 
"content of a context image that overlaps content of an earlier 
context image replaces the content of the earlier context image," 
as recited in Claim 14, or that "content of a context image that 
overlaps content of an earlier-displayed context image replaces 
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the content of the earlier-displayed context image, " as recited 

in Claim 19. Additionally, the Examiner's statement that "[t]he 

fact that images can be displayed, while being produced, suggests 

that an active image is produced in real time" is irrelevant to 

Claims 1 4 and 1 9 . 

Regarding Claims 21 and 22, the Examiner stated: 

Although Claims 21-22 do not disclose user interaction, 
or image designation, by a user, they do refer to 
displaying context images, in a specific manner. It is 
therefore implied, that these displays could not take 
place, without instruction, and are rejected, for the 
same reasons as Claims 5-12. 

Displaying an image in a specific manner does not in any way 
imply that such display could not take place without user 
instruction. The reasoning given in the Office Action for the 
rejection of Claims 5-12 is irrelevant to the patentability of 

i 

Claims 21 and 22. Claim 21 recites that !, all of the images 
displayed after a first display of a context image are displayed 
as context images" and Claim 22 recites that "less than all of 
the images displayed after a first display of a context image are 
displayed as context images." The Examiner has not addressed the 
particular limitations recited in Claims 21 and 22, and, in 
particular, has not shown how Kumar et al. teach or suggest such 
limitations . 

In view of the foregoing, it is requested that the rejection 
of Claims 1-24 under 35 U.S.C. § 102 be withdrawn. 
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New Claims 

Claim 25 has been added. Support for Claim 25 can be found 
in Applicants' specification at, for example, page 25, lines 13- 
18. Claim 25 depends directly on Claim 13 and indirectly on 
Claim 1 , and is therefore allowable over the teaching of Kumar et 
al. for at least the reasons given above with respect to those 
claims. Further, Kumar et al . neither teach nor suggest the use 
of a branch point to switch between active displays representing 
a panoramic region at different times, as in Claim 25. 



CONCLUSION 

Claims 1-24 were pending and were rejected. Claims 1, 23 
and 24 have been amended. Claim 25 has been added. In view of 
the foregoing, it is requested that Claims 1-25 be allowed. If 
the Examiner wishes to discuss any aspect of this application, 
the Examiner is invited to telephone Applicants' undersigned 
attorney at (408) 945-9912. 



I hereby certify that this correspondence is being 
deposited with the United States Postal Service as 
first class mail in an envelope addressed to: 
Assistant Commissioner for Patents, Washington, 
O.C. 20231, on November 30, 2001 . 

Date Signature 



Respectfully submitted , 

David R. Graham 
Reg. No. 36,150 
Attorney for Applicants 
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^ Version with Markings to Show Changes Made 

k <^dditions are underlined, deletions are enclosed in brackets) 



In the specification: 

The paragraph beginning at page 4, line 32 has been amended 
as follows: 

In another embodiment of the invention, one or more images 
are designated by a user and displayed as context image(s). For 
example, as active images are displayed, the user can designate 
particular active images to be context [image] images . 
Typically, in this embodiment, less than all of the images 
displayed after a first display of a context image are included 



in the context display. 

The paragraph beginning at page 17, line 2 has been amended 
as follows: 

The set of frames of visual data used to produce the visual 
display shown in FIGS. 1-3 (or, more generally, any set of frames 
of visual data used to produce a visual display according to the 
invention) can be used in a different manner to produce a visual 
display according to the invention that is similar to, but 
different [than] from , the visual display shown in FIGS. 1-3. 
Such a visual display includes a fixed context display that 

i 

includes content from all parts of the panoramic region from 
which images were obtained, i.e., the context display is produced 
from all of the frames of the set of frames of visual data. Such 
a context display can be produced by displaying content at each 



part of the context display that is the content from the 
temporally latest frame of visual data that includes content at 
that part of the panoramic region. (Alternatively, some other 
rule for mediating among overlapping frames of visual data could 
be used, such as using the content from the temporally earliest 
frame of visual data. ? ) Such a context display can also be 
produced by blending the overlapping parts of the frames of 
visual data, as discussed elsewhere herein. An active display is 
generated and displayed as discussed above, the active display 
appearing to be seamlessly integrated within a larger static 
panorama (the context display). 

The paragraph beginning at page 24, line 16 has been amended 
as follows: 

As illustrated in FIG. 9, context images generated from a 
set of frames of visual data that is different [than] from the 
set of frames of visual data being used to generate the active 
image remain as part of the visual display. (In FIG. 9, the 
context images to the left of location 901 and to the right of 
location 902 are from panoramic regions that are different [than] 
from the panoramic region from which the images, including the 
active image, between the locations 901 and 902 are taken.) 
However, a visual display in which transition is made at a branch 
point from one set of frames of visual data to another set of 
frames of visual data can be implemented in other ways. For 
example, switching from one set of frames of visual data to 
another set of frames of visual data may cause all images from 



the previous set of frames of visual data to be removed from the 
visual display. Or, in an embodiment of the invention in which a 
fixed context display is generated from all of the frames of a 
set of frames of visual data (as described above), switching to a 
new set of frames of visual data at a branch point can cause 
display of a new fixed context display that is generated from all 
of the frames of the new set of frames of visual data. 

In the claims: 

Claims 1, 23 and 24 have been amended as follows: 
1. (Amended) A, method for producing a visual display of a 
panoramic region from a set of temporally sequential frames of 
visual data, each frame of visual data representing an image 
defining a region within the panoramic region such that at least 
two of the images define regions that do not coincide, the method 
comprising the steps of: 

[generating a display of an] displaying a series of 
active [image] images generated from [each of] multiple 
temporally sequential frames of visual data; 

positioning each displayed image on a display screen 
with respect to the position of other displayed images on 
the display screen in accordance with the spatial 
relationship in the panoramic region of the content of the 
displayed image to the content of the other displayed 
images ; and 
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displaying one or more context images while an active 
image is displayed, each context image being at least part 
of an image represented by a frame of the set of temporally 

sequential frames of visual data. 

i 

23. (Amended) A system for producing a visual display of a 
panoramic region from a set of temporally sequential frames of 
visual data, each frame of visual data representing an image 
defining a region within the panoramic region such that at least 
two of the images define regions that do not coincide, the system 
comprising : 

means for [generating a display of an] displaying a 
series of active [image] images generated from [each of] 
multiple temporally sequential frames of visual data; 

means for positioning each displayed image on a display 
screen with resp'ect to the position of other displayed 
images on the display screen in accordance with the spatial 
relationship in the panoramic region of the content of the 
displayed image to the content of the other displayed 
images ; and 

means for [causing the display of] displaying one or 
more context images while an active image is displayed, each 
context image being at least part of an image represented by 
a frame of the set of temporally sequential frames of visual 



24. (Amended) A computer readable stprage medium on which 
is stored one or more computer programs for producing a visual 
display of a panoramic region from a set of temporally sequential 
frames of visual data, each frame of visual data representing an 
image defining a region within the panoramic region such that at 
least two of the images define regions that do not coincide, the 
one or more computer programs comprising: 

instructions for [generating a display of an] 
displaying a series of active [image] images generated from 

i 

[each of] multiple temporally sequential frames of visual 
data; 

instructions for positioning each displayed image on a 
display screen with respect to the position of other 
displayed images on the display screen in accordance with 
the spatial relationship in the panoramic region of the 
content of the displayed image to the content of the other 
displayed images; and 

instructions for displaying one or more context images 
while an active image is displayed, each context image being 
at least part of an image represented by a frame of the set 

i 

of temporally sequential frames of visual data. 

Claim 25 has been added as follows: 

25. (New) A method as in Claim 13, wherein: 

the set of frames of visual data in which a branch 
point is identified is a first set of temporally sequential 
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frames of visual data represent ing a panoramic region at a 
first time; and 

the new set 1 of frames of visual data is a second set of 
temporally sequential frames of visual data representing the 
panoramic region at a second time that is different from the 
first time. 
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